Heat of the Moment: Characterizing the Efficacy of Thermal Camera-Based Attacks

In this paper, we examine the potential of using a thermal camera to recover codes typed into keypads in a variety of scenarios. This attack has the advantage over using a conventional camera that the codes do not need to be captured while they are being typed and can instead be recovered for a short period afterwards. To get the broadest sense of how effective such an attack might be, we consider a number of variables: the material of the keypad, the user entering the code, the distance from the camera to the keypad, and the possible methods used to analyze the data. First, we present code recovery results from human review of our test data set; this provides us with a baseline for the overall effectiveness of thermal camera-based attacks. Second, using techniques from computer vision we automatically extract the code from raw camera data, thus demonstrating that this attack has the potential to scale well in practice. As we will see, both human and automated attacks are by and large successful in recovering the keys present in the code, even a full minute after they have been pressed; both methods are also able to determine the exact code (i.e., including the order in which the keys were pressed) for a smaller fraction of codes. Even without ordering, however, the search space of possible keys is still vastly reduced by knowing the keys pressed; for example, the search space is reduced from 10,000 possible codes to approximately 24 for a 4-digit code. In large-scale attacks involving many unique codes, such as on ATM PINs, our success rate indicates that an adversary can correctly recover enough codes to make such an attack economically viable.